LSTM Inefficiency in Long-Term Dependencies Regression Problems

Recurrent neural networks (RNNs) are an excellent fit for regression problems where sequential data the norm since their recurrent internal structure can analyse and process long. However, RNNs prone to phenomenal vanishing gradient problem (VGP) that causes network stop learning generate poor prediction accuracy, especially in long-term dependencies. Originally, gated units such as long short-term memory (LSTM) unit (GRU) were created address this problem. VGP was still is unsolved problem, even units. This occurs during backpropagation when weights tend vanishingly reduce hinder from correlation between temporally distant events (long-term dependencies), results slow or no convergence. study aims provide empirical analysis of LSTM with emphasis on inefficiency dependencies convergence because VGP. Case studies NASA’s turbofan engine degradation examined empirically analysed.